Appears in the Proceedings of the 15th International Conference on Computational Linguistics 
(COLING 94 ), Kyoto, Japan, August 1994, pp. 356 - 362 



DEFAULT HANDLING IN INCREMENTAL GENERATION 

Karin Harbuschf, Gen-ichiro Kikuif, Anne Kilgerf 

t ATRfl Fax: (+81 7749) 5 1308, E-mail: kikui@itl.atr.co.jp 
f DFKI, Fax: (+49 681) 302 5341, E-mail: harbusch|kilger@dfki.uni-sb.de 



OS 
OS 



o 
O 

o 

m 



> 

m 
o 
o 



> 

• 1—1 

X 
S3 



Abstract Natural language generation must 
work with insufficient input. Underspecifications 
can be caused by shortcomings of the component 
providing the input or by the preliminary state 
of incrementally given input. The paper aims to 
escape from such dead-end situations by making 
assumptions. We discuss global aspects of de- 
fault handling. Two problem classes for defaults 
in the incremental syntactic generator VM-GEN 
are presented to substantiate our discussion. 

1 MOTIVATION 

Natural Language Generation, i.e., the pro- 
cess of building an adequate utterance for 
some given content, is by nature a decision- 
making problem (Appelt, 1985). Internal de- 
cisions are made on the basis of the specified 
input. Unfortunately, input information can 
be insufficient in two respects: 

• If the input structure for generation is 
provided by another Al-system, global prob- 
lems in producing sufficient input information 
for the generator may occur, e.g., because of 
translation mismatches in machine transla- 
tion (Kameyama, 1991). In this case, the gen- 
erator either has to use a default or formulate 
a request for clarification in order to be able 
to continue its processing, i.e., to produce an 
utterance. During simultaneous interpreta- 
tion requests are rather unusual. Here de- 
faults allow for a standalone handling of the 
problem. For example, problems during 
speech recognition of automatic interpreta- 
tion can lead to results like "the (man/men) 
will come to the hotel tomorrow" . If the sys- 
tem is not able to give a preference for one 
of the alternatives, e.g., by evaluating con- 
text information, the generator has to choose 

*The author is currently at NTT Network Infor- 
mation Systems Laboratories (kikui@nttnly.ntt.jp). 



a probable number value on its own to com- 
plete verbalization. 

• Furthermore, for incremental generation, 
the input information is produced and handed 
over step by step, so that it can be temporar- 
ily incomplete — although as a whole it may 
become sufficient. This behaviour of a gen- 
erator is motivated by psycholinguistic obser- 
vations which show that people start speak- 
ing before all necessary linguistic material has 
been chosen (e.g., articulating a noun phrase 
before the dominating verb is selected). As a 
consequence of underspecification, incremen- 
tal generation is essentially based on working 
with defaults. Elements are uttered before 
the processing or input consumption has been 
finished. (Kitano, 1990) gives an example for 
defaults in the context of simultaneous inter- 
pretation: In Japanese, negation is specified 
at the end of the sentence while in English, it 
has to be specified in front of the finite verb. 
Therefore, during Japanese-English transla- 
tion, where analysis, transfer, and generation 
are performed in a parallel and incremental 
way, the system has to commit, e.g., positive 
value before knowing the actual polarityQ. 

Generally speaking, default handling speci- 
fies how processing, i.e., further decision-mak- 
ing, can continue without sufficient input in- 
formation. So, one can compare default han- 
dling with advice to the system. For reasons 
of uncertainty of assumptions, incremental 
systems with this facility must be able to re- 
pair the default decision when the assump- 
tion turns out to be wrong by information 
given later. Catching on to the above exam- 
ple, there can be a negation specifier given at 
the end of the Japanese input sentence which 



1 Alternatively, the system could use the dialogue 
context to infer a negation value +/-. 



cannot be simply integrated into the output 
sentence because the finite verb has already 
been uttered. In this case, the output has to 
be repaired, e.g., by repeating parts of the ut- 
terance: "I will be able to meet you . . . oops 
... I won't be able to meet you at the hotel 
this evening." 

In the following sections, we argue for the 
appropriateness of processing-conforming de- 
fault handling. Basically, the processing-con- 
forming mode makes the overall system ho- 
mogeneous because the combination of 
default-caused processing and input-licensed 
processing requires no specific description. 
The homogeneity becomes especially helpful 
in the case where the input verifies the de- 
fault assumption rendering unnecessary any 
recomputation. For the opposite case where 
the default must be withdrawn we have to 
mark all defaults. Even more homogeneity 
is introduced to an incremental system if the 
default descriptions are given in terms of in- 
put specifications. This representation allows 
for easy checking the coincidence between a 
chosen default and input given later. 

The content of this paper can be summa- 
rized as follows. Section 2 provides a general 
description for defaults in generation empha- 
sizing the specific requirements in an incre- 
mental system. After identifying the condi- 
tions under which defaults are triggered (sec- 
tion 2.1), the application of a default (sec- 
tion 2.2) and the definition of its description 
(section 2.3) is outlined. The crucial case of 
removing defaults not coinciding with newly 
arriving input in an incremental system is dis- 
cussed in section 2.4. 

In section 3, this mechanism is applied to 
the incremental sentence generator VM-GEN. 
In the beginning of the section, the basic de- 
sign of the system is outlined. Later on, de- 
fault handling is included and exemplified for 
two general cases. 

In the final section we summarize the main 
results of the paper. Furthermore, we dis- 
cuss how default handling can be adapted 
to multilingual generation, as required by the 
speech-to-speech translation system VERB- 
MOBIL (Block et al, 1992). 



2 GENERAL DISCUSSION OF 
DEFAULTS 

In the literature of non-incremental genera- 
tion, the need for defaults is hardly ever taken 
into account. The common point of view re- 
stricts the input to be sufficient for genera- 
tion (see, e.g., the Text Structure by (Meteer, 
1990) for a syntactic generator). In incremen- 
tal generation, most authors agree on the ne- 
cessity of using defaults (see, e.g., (De Smedt, 
1990; Kitano, 1990; Ward, 1991)). Neverthe- 
less, they do not in sufficient depth answer 
the question of how to guide the processes of 
default handling and repair within a gener- 
ator. This problem is the starting-point for 
the following considerations. 

We assume that generation is a decision- 
making process with the aim of producing a 
plausible utterance based on given informa- 
tion. As mentioned in section 1, there are 
cases where this process stops (caused by un- 
derspecification of the input) before finishing 
its output. 

We define a module named default han- 
dler which tries to resume the process by giv- 
ing advice to it, i.e., by making assumptions 
about the missing input specification. With 
respect to this task it is discussed 

1. in which situations defaults are applied (see 
section 2.1), 

2. how default handling is integrated into a 
system (see section 2.2), 

3. how the knowledge for default handling is 
described (see section 2.3), and 

4. how assumptions are cancelled when they 
turn out to be inconsistent with newly ar- 
riving input (see section 2.4). 

In incremental generation, as mentioned in 
section 1, interleaved input consumption and 
output production causes specific default sit- 
uations. An incremental processing scheme 
allows for an increase of efficiency and flexi- 
bility, e.g., by making the analysis and gen- 
eration processes of a system for simultane- 
ous interpretation overlap in time. There are 
two competing goals of incremental genera- 
tion for spoken output, that must be taken 



into account when estimating the usefulness 
of defaults: 

Fluency: Long hesitations should be avoided 
during the production of an utterance, in 
order to be acceptable to the hearerQ 

Reliability: Errors in an utterance may 
cause misunderstanding. In most cases, er- 
rors should be recovered by appropriate self- 
corrections]]. Excessive use of self-correc- 
tions or erroneous expressions should be 
avoided because they decrease intelligibil- 
ity of the utterance. 

Obviously there is a trade-off between fluency 
and reliability: maximal reliability requires 
'secure' decisions and therefore leads to out- 
put delay. On the other hand, maximal flu- 
ency necessitates the use of assumptions and 
repair, respectively. 

2.1 When to Trigger Default Handling 

We define as default situation the situation 
where a generation system has not yet fin- 
ished the utterance but at the same time has 
consumed all given input and is not able to 
continue processing. In non-incremental gen- 
eration, this corresponds to the fact that the 
input lacks necessary information, because 
the entire input is assumed to be given at 
one time (e.g., the undecidable number value 
of the example described in section 1). Thus, 
default handling should be triggered immedi- 
ately. 

In incremental generation, however, the sys- 
tem may get a new piece of information later 
on that enables it to continue processing (e.g., 
the specification of a negation value + as out- 
lined in the example in section 1). There- 
fore, possible alternatives are either to wait 
for the next input or to trigger default han- 
dling. The former violates the fluency goal, 
the latter may violate the reliability goal. We 
propose the explicit use of time-limits for de- 
lay interval^. 

2 Humans often fill such pauses with fillers like "er" 
or "what shall I say" . 

3 Sometimes, correction is unnecessary if (the 
speaker believes that) the hearer can infer the in- 
tended utterance from erroneous speech. 

4 An explicit parameter expressing the desired de- 
gree of fluency influences the time-limits. 



Furthermore, the certainty of a default is 
described by a value. As soon as a default 
situation is identified, the certainty of the de- 
fault is checked to see whether it exceeds a 
predefined threshold that determines the de- 
gree of fluency/reliability^. 

Each application of a default decreases the 
global certainty of the system's state. Conse- 
quently, there should be a limit for the maxi- 
mal number of defaults applicable to the same 
sentence. 

2.2 How to Integrate Default Handling 

Basically, there are two strategies to integrate 
default handling into ongoing processing. 

Defaults may be handled in a way that dif- 
fers from the 'normal' processing of the sys- 
tem, e.g., as short-cuts. One advantage can 
be an efficient handling of defaults. Further- 
more, the designer of the default component 
is completely free in deciding about the re- 
alization of defaults in the system. A dis- 
advantage is the difficulty of providing con- 
sistency between default-caused and input- 
licensed processing. 

Alternatively, the ongoing processing can 
deal with the default values in an ordinary 
manner (processing-conforming default han- 
dling). This may be less efficient but guaran- 
tees consistency during processing, especially 
in case of a replacement by an input-licensed 
value. For incremental generation, the sys- 
tem has to provide repair facilities in any case. 
So, they can also be used for non-monotonic 
modifications of default-caused results. We 
take this option in order to make the overall 
system homogeneous. 

2.3 How to Describe Defaults 

The knowledge source that is used for default 
handling should provide the most plausible 
actions for a default situation. We represent 
the knowledge set of heuristic rules called 
default descriptions. A default description de- 
fines a set of operations that should be carried 

5 The basis for assigning certainty values to de- 
faults should be a corpus study that can be used to 
find statistical evidence for various features with al- 
ternative values (like number, voice, . . . , see, e.g., 
(Bock and Warren, 1985)). 



out in a certain situation where the genera- 
tion process can not be continued. A default 
description has the following form: 

default _ default _ certainty 

preconditions body ' value 

The set of default preconditions defines tests 
that are applied to the given situation in or- 
der to find out whether the corresponding de- 
fault body can be activated. They include 
tests for the existence of particular informa- 
tion, tests for the structure under creation 
and tests for the state of processing. 

The default body describes how to continue 
processing with defaults in an adequate way. 
For incremental systems, we propose to ex- 
press the body specification of input in- 
crements. An important prerequisite is that 
the size of increments is defined flexibly enough 
to cope with varying amounts of information. 
Obviously, an important advantage of this ap- 
proach is homogeneity of the overall system. 
Especially, the homogeneous representation 
of default-caused and input-licensed struc- 
tures is the easiest and most direct way to 
test coincidences or contradictions between 
default-specified and input-caused values. In 
section 3, this approach is outlined by differ- 
ent examples. For non-incremental systems, 
an operational approach is preferable since 
there is no way to consume additional input 
increments, presupposing that the input has 
been considered as a whole before a default 
situation occurs^. 

If several default preconditions are appli- 
cable, the certainty values for default descrip- 
tions are examined to find which provides the 
system with the most plausible action. 

The individual default descriptions should 
take into account the global constraints for 
processing stated in the knowledge sources of 
the system. For example, the assumption of 
nominative case for a German NP comple- 
ment can regularly be made only once for the 

6 The difference between incremental and non- 
incremental generation becomes smaller, if we assume 
that defaults in a non incremental system can be trig- 
gered after the system has only considered parts of its 
input information. In this case, the input considered 
after default handling becomes comparable to later 
increments. 



same verb. For reasons of homogeneity, the 
default description should at least be compat- 
ible with the specifications of the knowledge 
used for basic processing. In order to guar- 
antee consistency, default descriptions should 
merely contain what is orthogonal to the ba- 
sic knowledge sources. 

2.4 How to Cancel Defaults 

The repair of false assumptions is a crucial 
point of default handling in the context of in- 
cremental processing because the default in- 
formation does not remain locally but can 
cause further decisions of the system. Con- 
trarily, for non-incremental input there will 
be no value given that can contradict default 
values. 

As a first step of repair, inconsistencies be- 
tween input-provided and default-caused val- 
ues are identified by simply matching the val- 
ues. Then effects of the respective defaults 
are withdrawn introducing the input-provided 
values into the system. Generally, a deci- 
sion during generation influences other deci- 
sions all over the system. Thus the effect of 
a default body may be propagated through 
the entire system (e.g., choosing a construc- 
tion of main clause with causal subordinate 
clause influences the choice of syntactic real- 
izations). 

Roughly speaking, withdrawing a default 
assumption can be realized by backtracking 
to the earlier state of the system where the 
default had been introduced or by non-mon- 
otonic changes to the current state of the sys- 
tem. The disadvantage of backtracking is that 
partial results are thrown away which could 
be reused during further processing. Non- 
monotonic changes preserve these results. In 
this framework, cancelling defaults requires 
the system to identify which results are caused 
by default handling. Dependency links be- 
tween the immediate result of a default body 
and results of the influenced decisions allow 
for this identification. The disadvantage of 
non-monotonic changes is the complexity of 
computation, e.g., supported by a truth main- 
tenance system. When designing an incre- 
mental system, simple backtracking is ruled 



out because the part of the sentence uttered 
cannot be withdrawn after it has been per- 
ceived by the addressee of the message^]. 

So, we end up with a processing-conform- 
ing default handler for generation realizing re- 
pair by non-monotonic changes. 

3 EXAMPLES OF DEFAULTS 
IN VM-GEN 

The adaptation of our general discussion of 
default handling to the system VM-GEN not 
only provides concrete examples for the reader 
but also shows that a homogeneous combina- 
tion of default handling, regular processing, 
and utterance repair is possible. 

The syntactic generator VM-GEN is a fur- 
ther development of TAG-GEN (Kilger, 1994) 
within the framework of VERBMOBIL, a 
speech-to-speech translation system. Its use- 
fulness for simultaneous interpretation results 
from its incremental and parallel style of pro- 
cessing. VM-GEN is able to consume input 
increments of varying size. These increments 
describe lexical items or semantic relations 
between them. Single input increments are 
handed over to objects of a distributed paral- 
lel system, each of which tries to verbalize the 
structure that results from the corresponding 
input increment. VM-GEN uses an exten- 
sion of Tree Adjoining Grammars (TAGs, cf. 
(Joshi, 1985)) as its syntactic representation 
formalism that is not only adequate for the 
description of natural language but also sup- 
ports incremental generation (Kilger and Fin- 
kler, 1994). 

In the following, we introduce examples for 
default processing triggered during the Ger- 
man inflection process in VM-GEN to sub- 
stantiate the global statements made in sec- 
tion 2. Inflection uses some syntactic proper- 
ties of an element to compute its morpholog- 
ical form. This information has partly to be 
specified in the input (e.g., the number for a 
noun) and is partly inherited from other el- 

7 If some phrases influenced by defaults have al- 
ready been verbalized, the effect of verbalization can 
be cancelled by using repair words like "oops" or 
"sorry" when starting the modified utterance. 



ements (e.g., the number for a verb or the 
case for a noun) . The two reasons for missing 
information necessitate different methods of 
treatment which nevertheless both can uni- 
formly be integrated into regular processing. 

If information of the first type is missing 
(e.g., because of problems during analysis, see 
section 1), an assumption can be made locally 
by simulating the respective part of the input. 
The default for missing number information 
in VM-GEN would look as follows: 

(cat(OBJ)=N) (ENTITY OBJ . g g | 

(number (OBJ)=N I L) (number sg)) ' 

The set of default preconditions is applied 
to alfl objects (OBJ) of VM-GEN in order 
to test the kind of underspecification ('num- 
ber' in the example). The default body intro- 
duces a new value (sg) by creating an input 
increment default. The test for coinci- 
dence with the input-licensed value is realized 
by a comparison in the objects of VM-GEN. 
There is a unique association of input incre- 
ments and objects of VM-GEN (OBJ is used 
as identifier) that allows for translating an in- 
put modification into a modification of the 
state of the respective object. In case of con- 
tradictions the default and all default-caused 
decisions are revised0 (see below). 

Making an assumption can be influenced by 
global constraints. An example, which is well 
studied in psycholinguistics, is the utterance 
of a noun before the verb has been chosen. 
If, e.g., the noun "Besucher" (English: "visi- 
tor") is known to be the agent of an action, 
it may be uttered as subject in the first posi- 
tion of the sentence by default. This treat- 
ment presupposes the choice of a 'dummy' 
verb, which at least sub categorizes a subject 
and has active voicef^. The use of a dummy 

8 'ENTITY' introduces information about a lexical 
item. For reasons of incrementality, there may be 
several ENTITY-packages specified for the same item 
which are composed to receive the global information. 
For certainty values, we use values between and 1, 
where 1 means high reliability. 

9 In the actual implementation we preselect candi- 
dates with missing values for reasons of efficiency. 

10 For ongoing work on repair in VM-GEN see (Fin- 
kler, 1994). 

11 This kind of expansion is called "provisional up- 
ward expansion" by (De Smedt, 1990). 



verb and an underspecified verbal structure 
the NP is integrated into allows for a simple 
global test that rules out the same case value 
assignment to different NP complements as 
it is required for most of the German verbs. 
This rule is represented in the grammar as 
a part of the description of subcategorization 
frames for verbs. For reasons of homogene- 
ity we use the information stored in the syn- 
tactic knowledge sources of VM-GEN for ex- 
pressing syntactic constraints during default 
handling as well. The advantage of this ap- 
proach is, that processing is continued in a 
consistent way, which eases the introduction 
of the input-licensed value. One default for 
choosing a missing case-value is specified as 
follows: 

(ENTITY OBJ' 
(cat(OBJ)=N) (CAT v) 

(case(OB,J)=NIL) _^ (VOICE active)) ^ g pi 

(function(OBJ)=agent) (RELATION REL' ' >— ' 
(lemma(head(OBJ))=NIL) (HEAD OBJ') 

(MODIFIER, OBJ)) 

The default preconditions of the rule charac- 
terize a situation where an object (OBJ) con- 
tains no information about the case but iden- 
tifies the input category as 'N' for noun. Fur- 
thermore, the semantic function of the object 
is specified as 'agent' but no verb defined yet 
(lemma(head(OBJ))=NIL) in the head ob- 
ject. That is why, the N-object cannot inherit 
value and also does not know whether 
it is allowed to occupy the front position in 
the utterance. 

Evaluating the default body, the system 
creates a V-object OBJ'. On the basis of the 
input information in (ENTITY OBJ' . . . ) it 
chooses a minimal syntactic structure from 
the inheritance net of the grammar, that just 
desribes a verb category without concrete filler 
(a dummy verb) plus a subject complement 
and active voice for the verbal phrase. Now, 
the N-structure is combined with the V-struc- 
ture of the introduced V-object as during nor- 
mal processing. Therefore, the case value can 
be inherited. Additionally, the first position 

12 'RELATION' introduces the specification of a re- 
lation between two lexical items which are identified 
by the names of their objects. 



can be assigned to the subject which can be 
uttered now. 

The basic VM-GEN module provides re- 
pair strategies in order to allow for the spec- 
ification of additions, modifications and dele- 
tions of input increments, i.e., to model a 
flexible input interface. Three features of the 
system are basically used for repair: First, in- 
put increments are uniquely associated with 
objects of VM-GEN, so that input modifica- 
tions can be translated into modifications of 
the objects' states. Second, each modification 
of an object's state makes it compare new and 
old information. In case of a difference, the 
modified parts are sent to all concerned ob- 
jects. Third, the dependency relations that 
determine the communication links between 
objects allow for a hierarchical organization 
of the objects, which is the basis for synchro- 
nizing repair. 

A repair must be triggered in the exam- 
ple described above if, e.g., a verb with voice 
passive is actually specified. In this case, the 
mapping of the semantic role 'agent' to the 
syntactic function 'subject' is revised. The 
agent now has to be realized as part of a 
"von" -phrase, e.g. "dieser Termin wird von 
dem Besucher gewiinscht." (word-for-word: 
"this date is whished by the visitor (dative ob- 
ject)"). Furthermore, the object checks 
whether the previously uttered part of the 
sentence includes some of the revised mate- 
rial (i.e., whether the object itself has par- 
ticipated in uttering). If this is the case, it 
sends an error message up to the uppermost 
object of the hierarchy that actually is en- 
gaged in uttering. This object is able to syn- 
chronize global repair. Up to now, we just 
realized a simple repair strategy that consists 
of repeating the concerned parts of the ut- 
terance, e.g. "der Besucher ... ah ... dieser 
Termin wird von dem Besucher gewiinscht" . 

4 DISCUSSION 

This paper proposes a processing-conforming 
default handler for generation realizing repair 
by non-monotonic changes. We provide the 
system with default descriptions. The set of 
default preconditions expresses possible rea- 



sons for dead-end situations. A default is 
triggered, if the preconditions match the cur- 
rent situation and the certainty value of the 
default exceeds the predefined threshold. The 
default body is expressed in terms of the miss- 
ing input specification in order to make the 
system work homogeneously. We have veri- 
fied the advantages of processing-conforming 
default handling by implementing a default 
handler for VM-GEN. 

As future work, we will extend the default 
preconditions towards handling complex con- 
textual information. We will apply default 
handling to microplanning and lexical choice 
within VERBMOBIL. With respect to a so- 
phisticated output, we aim to combine VM- 
GEN with a flexible repair component. 

The system VM-GEN is used in the VERB- 
MOBIL scenario for multilingual generation 
(English, German, and Japanese). We mean 
by multilinguality that the same processing 
is applied for different languages. In the un- 
derlying knowledge sources language-specific 
constraints are defined. Default handling can 
be easily adapted to the requirements of mul- 
tilingual generation by using language-specif- 
ic default-descriptions. 

For all knowledge sources the question arises 
how knowledge can be shared. We intend 
to use core knowledge sources for represent- 
ing common phenomena. The core set of de- 
fault descriptions for English and German, 
e.g., contains the description of a reaction to 
a missing number value for a noun. We aim 
to develop an efficient storing mechanism us- 
ing a hierarchy of locally intersecting core de- 
scriptions. 
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